Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix: Make peers sync secondary index #2390

Merged
merged 6 commits into from
Mar 7, 2024

Conversation

islamaliev
Copy link
Contributor

Relevant issue(s)

Resolves #2379

Description

Make peers updated update local storage when an update happens on docs.

@islamaliev islamaliev added bug Something isn't working area/datastore Related to the datastore / storage engine system area/p2p Related to the p2p networking system labels Mar 7, 2024
@islamaliev islamaliev added this to the DefraDB v0.10 milestone Mar 7, 2024
@islamaliev islamaliev requested a review from a team March 7, 2024 18:22
@islamaliev islamaliev self-assigned this Mar 7, 2024
client/db.go Outdated Show resolved Hide resolved
Copy link

codecov bot commented Mar 7, 2024

Codecov Report

Attention: Patch coverage is 50.00000% with 57 lines in your changes are missing coverage. Please review.

Project coverage is 75.00%. Comparing base (4e9f6f4) to head (ed5ec47).

❗ Current head ed5ec47 differs from pull request most recent head b89c0a2. Consider uploading reports for the commit b89c0a2 to get more accurate results

Additional details and impacted files

Impacted file tree graph

@@             Coverage Diff             @@
##           develop    #2390      +/-   ##
===========================================
+ Coverage    74.99%   75.00%   +0.01%     
===========================================
  Files          268      268              
  Lines        25912    26003      +91     
===========================================
+ Hits         19431    19502      +71     
- Misses        5170     5177       +7     
- Partials      1311     1324      +13     
Flag Coverage Δ
all-tests 75.00% <50.00%> (+0.01%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

Files Coverage Δ
client/index.go 80.95% <ø> (ø)
http/client.go 52.84% <100.00%> (ø)
db/txn_db.go 63.30% <50.00%> (ø)
db/index.go 67.59% <50.00%> (-0.33%) ⬇️
http/client_collection.go 31.14% <0.00%> (-0.57%) ⬇️
net/server.go 69.48% <55.32%> (-1.82%) ⬇️
db/collection_index.go 85.00% <49.06%> (-5.85%) ⬇️

... and 5 files with indirect coverage changes


Continue to review full report in Codecov by Sentry.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 4e9f6f4...b89c0a2. Read the comment docs.

client/db.go Outdated Show resolved Hide resolved
Copy link
Contributor

@AndrewSisley AndrewSisley left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good, but I think there are few bits that need sorting out before merge

db/collection_index.go Outdated Show resolved Hide resolved
db/txn_db.go Outdated Show resolved Hide resolved
http/client.go Outdated Show resolved Hide resolved
tests/clients/cli/wrapper.go Outdated Show resolved Hide resolved
@islamaliev islamaliev requested a review from AndrewSisley March 7, 2024 21:37
CreateDocIndex(context.Context, *Document) error

// UpdateDocIndex updates the index for the given document.
// WARNING: This method is only for internal use and is not supposed to be called by the client
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

praise: These are suitably scary - thanks 😁

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I second that 👍

if err != nil {
return err
}
err = c.indexNewDoc(ctx, txn, newDoc)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nitpick: I do see delete+new as still being business logic, but only at a nitpick level - so feel free to merge as is :)

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can you expand on what is your nitpick concern here Andy?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I have none and thought I was in the txn_db.go file :)

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah I see I how I got confused - they have moved to collection, and I new that they had moved, but my brain still thought of the implementation as being in the txn_db file when I saw the txn logic. I wonder if we can do a similar job to collection that we did to client.Store and split out the txn boilerplate to another file sometime

Copy link
Collaborator

@fredcarle fredcarle left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM!

return nil, client.NewErrCollectionNotFoundForSchema(schemaRoot)
}
var col client.Collection
for _, c := range cols {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

praise: Thanks for moving this somewhere out of the way :)

Copy link
Contributor

@AndrewSisley AndrewSisley left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM - thanks Islam :)

@islamaliev islamaliev merged commit b8d9803 into sourcenetwork:develop Mar 7, 2024
27 of 28 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
area/datastore Related to the datastore / storage engine system area/p2p Related to the p2p networking system bug Something isn't working
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Secondary indexes does not appear to work with P2P
3 participants